Docs installation improvements #12304
Conversation
|
The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. |
|
Love it ❣️ |
|
Same ❤️ |
|
The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. |
…for installation docs
…ements for installation docs
|
The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. |
… Improvements for installation docs
| fetch_and_cache(script_url, output_filename) | ||
|
|
||
|
|
||
| def build_finished(app, exception): |
There was a problem hiding this comment.
I download the necessary files at the start of the documentation build process to avoid a problem with snowflake-connector-python.
See: snowflakedb/snowflake-connector-python#324
dev/package-data/providers.yaml
Outdated
| Amazon integration (including `Amazon Web Services (AWS) <https://aws.amazon.com/>`__). | ||
| python-module: airflow.providers.amazon | ||
| versions: | ||
| - 1.0.0b1 |
There was a problem hiding this comment.
I think what we really want to do instead of keeping it in one file, this should be a separate provider.yaml in each provider's directory, rather than one big file with all providers. What we really want to do is make sure that ther are NO specific references to specific providers in a core part. This should be all automatically discoverable. It should be enough to drop the "provider.yaml" in "providers/google" package and the script should find that file and get provider information from there.
This has the added benefit, that all changes to a provider (including version change) will be done in the right "provider/nnn" directory and we will be able to see it in the list of changes generated there. Also this means that if you will be changing provider and updating it's description or version, there will be no "core" change whatsoever - which means that the "core" tests will not be triggered.
I am happy to approve it now and change it as a follow-up PR if you do not want to change it now.
There was a problem hiding this comment.
This file is not part of the core. This is only part of the metadata about the project. When we split the documentation into several packages, this document is no longer used by any documentation and only used by other tools such as website building to get the current package status.
However, I can try to change it as a follow-up PR.
There was a problem hiding this comment.
I've thought about it a bit longer and I'll try to do it now because it should be automated quite simply.
There was a problem hiding this comment.
I updated the PR. Now we have a separate provider.yaml file for each provider.
.pre-commit-config.yaml
Outdated
| @@ -245,11 +245,11 @@ repos: | |||
| pass_filenames: false | |||
| entry: ./scripts/ci/pre_commit/pre_commit_check_order_setup.py | |||
| - id: setup-installation | |||
… fixup! Improvements for installation docs
… fixup! fixup! Improvements for installation docs
… fixup! fixup! fixup! Improvements for installation docs
… fixup! fixup! fixup! fixup! Improvements for installation docs
|
The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. |
… fixup! fixup! fixup! fixup! fixup! Improvements for installation docs
|
The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. |
… fixup! fixup! fixup! fixup! fixup! fixup! Improvements for installation docs
* Improvements for installation docs
I've improved the documentation a bit by separating the reference documentation from the concept documentation. I also added reference documentation for providers along with Python API reference for them i.e. i.e. you can see what is available in each pip package.
All data is in a separate YAML file (along with JSON Schema validation), which allows us to reuse this data, for example to build README.md files.
Preview:
https://zany-property.surge.sh/provider-packages-ref.html
https://zany-property.surge.sh/extra-packages-ref.html
https://zany-property.surge.sh/provider-packages.html
https://zany-property.surge.sh/installation.html
Close: #12281
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.